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Abstract 


'We  sliuw,  for  convex  optimization  ln(^^,  how  a minor  modification  of 
A 

the  usual  Lagrangean  functlo^ (unlike  that  of  the  "augmented  Lagrangeans") , 
->  plus  a limiting  operation,  allows  one  to  close  duality  gaps  even  in  the 
absence  of  a Kuhn-Tucker  vector,  (see  the  introductory  discussion,  and  the 
discussion  in  Section  4 up  to  equation  (50)) The  cardinality  of  the 
convex  constraining  functions  can  be  arbitrary  (finite,  countable,  or 


uncountable). 


reveals  much  finer 


In  fact,  -mae-  main  result 

> A . 

detail  concerning  tmr  ■'limiting  Lagrangean.*^  There  are  affine  minorants 

"tkijr.x  ^ ^ I 

(for  any  value  0 < 9 <~l)of  the  limiting  parameter  d)  of  the  given  convex 
functions,  plus  an  affine  form  nonpositive  on  K,  for  which  a general 

Ay- 

linear  inequality  holds  on^*^.  After  substantial  weakening,  this  inequality 
C- 

leads  to  the  conclusions  of  the  ^evlous  paragraph. 

This  work  is  motivated  by,  and  is  a direct  outgrowth  of,  research 
joint  with  R.J.  Duffin,  which  is  cited  as  our  reference  [6]. 
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A Limiting  Lagrangean  for  Infinitely-Constrained 
Convex  Optimisation  In 

by  K.C.  .luros  low* 

In  Honor  of  Dick  Duffln 

We  consider  convex  programs  (see  Section  2 below) 

inf  l^(x) 

(CP)  subject  to  fjj(x)  <0,  heH, 

X e K- 

■ 'I 

with  possibly  infinitely  many  constraints,  and  show  under  a weak  constraint 
qualification  (CQ)  (see  below)  that  a small  modification  of  the  ordinary 
Lagrangeun  always  closes  the  duality  gap.  To  be  more  specific  (see 
Corollary  3 below)  we  show  there  are  scalars  w^,  Wj^  and  a vector  weR 
such  that,  if  0 < 0 < 1,  there  are  nonnegative  scalars  {A.^^  | h cH}  with 

(DE)  (1  + 9wQ)fQ(x)  + 0(wx  + Wj,)  + 2 Xj^fj^(x)  > v(P) 

heH 

for  all  X « K,  where  v(P)  is  the  value  (assumed  finite)  of  (CP).  The 
summation  in  (DE)  is  never  problematic,  since  only  finitely  many  are 

i 

non-zero. 

Our  constraint  qualification  (CQ)  does  not  imply  the  existence  of  a I 

Kulm-'fucker  vector,  and  hence  is  weaker  than  the  usual  ones  (see  Section  2 | 

below).  Thus  (DE)  places  many  duality  gaps  in  a simple  perspective:  the 
criterion  function  £q(x)  should  not  be  weighted  by  unity,  but  rather  by  a 
number  arbitrarily  near  unity;  and  then  an  affine  linear  "compensation" 

wx  -I-  Wj^  is  needed,  but  it  can  be  weighted  by  any  positive  amount,  however 

. ■ i 

small. 

i 

^Address  after  September,  1978:  College  of  Industrial  Management,  Georgia 
Institute  of  Technology,  Atlanta,  Georgia,  30332. 
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Our  methods  of  proof  can  be  succinctly  described,  and  were  developed 
jointly  in  [ 6 I . We  reason  as  follows.  Since  closed  convex  sets,  and  the 
epigraphs  of  closed,  convex  functions,  are  describable  by  infinitely  many 
linear  inequalities  — in  the  terminology  of  Chames , Cooper,  and  Kortanek 
[3  ],  they  are  describable  as  a "semi -infinite"  constraint  set  — this 
convex  optimization  ought  to,  in  principle,  be  reducible  to  results  about 
semi-infinite  systems. 

Recently,  R.J.  Duffin  and  the  author  [ 6 ] found  methods  of  reducing 
convex  programs,  under  a constraint  qualification,  to  semi-infinite  programs, 
of  applying  the  "appropriate"  result  on  semi-infinite  systems,  and  then  re- 
interpreting the  resulting  conclusion  (which  is  a conclusion  about  the 
semi-infinite  program)  as  a conclusion  about  the  convex  program. 

This  paper  is  very  much  a replication  of  [ 6 ] , except  that  a different 
result  about  semi-infinite  programs  is  (first  established  here  in  Section  1 
and  then)  applied,  and  then  a different  conclusion  about  the  Lagrangean  is 
obtained . 

For  related  work,  see  Blair's  generalization  [2  , Theorem  3 I of  a 
result  from  an  early  draft  of  I 6 1 , which  we  quoted  to  him,  as  well  as 
Mchlnden's  further  generalization  of  this  result  | 8 ] to  certain  infinite- 
dimensional spaces.  McLinden's  work  [8  1 uses  the  elegant  theory  of 
conjugates  of  convex  functions,  as  developed  by  Rockafellar  in  [9  1.  The 
paper  (4J  is  also  relevant. 


Section  1:  A Strengthening  of  a Corollary 


of  Blair’s  "Ascent  Ray"  liieoretn 


In  this  section,  we  strengthen  [ 1 , Corollary  2 1 to  a form  which,  as 
we  show  below,  actually  implies  the  main  result  [ 1 , Iheorem]  of  [ 1 ] . 

Let  cone(S)  resp.  clcone(S)  denote  the  cone  resp.  the  closure  of  the 
cone  spanned  by  S (see  [9l).  The  following  result  is  well-known  (see, 
e.g.,  t 9 ]),  and  is  a direct  application  of  the  Separating  Hyperplane  Theorem. 

Lenma  1:  For  I ^ 0 an  arbitrary  index  set,  indexing  a set  of  vectors 
(a^  I i €l}  in  r”,  suppose  that 

a^x  > 0,  all  i e I 
implies  cx  > 0 

for  any  x e R • 

Then  c e clcone  ({a^  ] i « l})  . 

Lemma  2:  For  1 / 0 an  arbitrary  index  set,  suppose  that 

a^x  >0»  iel 

implies  cx  > 0. 

Then  there  is  a vector  w with  the  following  property: 

For  any  0,  0 < 0 < 1,  there  is  a set  of  nonnegative  multipliers 
I i el},  only  finitely  non-zero,  such  that 

(1)  c + 9w  = Z X a^, 

iel 

In  fact,  if  V is  any  point  in  the  relative  interior  of  the  set 


(2) 


C^  = cone  ({a^  | lei}) 


4 


we  may  set 

(3)  w = V - c , 

Proof . By  Lemma  1,  c e cl  , and  since  v is  in  the  relative  interior  of  d , 
then  0 < 9 < 1 implies  that  9v  + (1  - 9)c  is  in  the  relative  interior  of  d , 
by  the  accessibility  lemma  [ 9 ] ; and  hence  can  be  expressed  in  the  form  of 
the  right -hand -side  of  (1)  with  | i el}  a finitely  non-zero  set  of 
multipliers.  However, 

9v  + (1  - 9)c  = c + 9(v  - c)  = c + 9w 

and  so  (1)  holds. 

Since  any  convex  set  has  a relative  interior,  at  least  one  such  w 
given  by  (3)  exists. 

Q.E.D , 

We  now  give  our  strengthening  of  [ 1 , Corollary  2],  which  is  closely 
related  to  Kortanek's  "perfect  duality"  results  [ 7 ]. 

Tlieorem  1:  Let  I 0 be  an  arbitrary  index  set,  and  suppose  that  the  system 

(4)  a^x  > b , all  i e I 

has  a solution  in  r". 

Suppose  also  that  (4)  implies 

(5)  cx  > d 
for  any  x c R*^ . 

Then  there  is  a vector  w e r"  and  a scalar  w^eR,  with  the  following 


property: 


k 


For  every  0 < 9 < 1 there  are  nonnegative  scalars  | i cl),  only 
finitely  non-zero,  which  satisfy 

(6)  c + 0w  = E A.  a^ 

iel  ^ 

(7)  d + 9w„  < E X b . 

iel  ^ ^ 

In  fact,  if  (v,-Vq)  is  any  point  in  the  relative  interior  of  the  set 

(8)  C"  = cone  ({ (a^ , -b . ) | i e l)  U [ (0, 1)} ) 
we  may  set 

(9)  (w,-Wq)  = (v,-Vq)  - (c,-d) 

i .e. , w = V - c and  Wq  = Vq  - d . 

Proof : Since  (4)  is  consistent,  and  (4)  implies  (5),  one  easily  proves  that 


a^x  >0,  all  i e I 

implies 

cx  ■ 0. 

Therefore  for 

(x,r)eR^^  (x  c r")  arbitrary. 

r > 0 

a^x  - b.r  > 0 for  all 

1 — 

implies 

cx  - dr  >0, 

We  apply  Lemna  2 to  reach  the  conclusion,  that  there  exists  (w,-Wq)  c 
weR*',  with  the  following  property: 

For  any  0 < 0 < 1,  there  are  non-negative  scalars  | iel),  finitely 
non-zero,  and  a scalar  <p  > 0,  such  that 
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I 

I 


(c,-d)  + 9(w,-Wq) 


cp(0,l)  + S ), 

iel 


Also,  if  (v,-Vq)  is  any  point  in  the  relative  interior  of  c',  we  may 
set  (9). 

Now  analyzing  (10)  by  components  gives  (6)  and  (7). 

Q.E.D. 

To  compare  our  results  (6),  (7)  with  the  conclusions  of  (1),  (ii), 

(iii)  [ 1 , Corollary  2],  divide  both  sides  of  [ 1 , Corollary  2(i)]  by 
\n  > 0 (which  can  be  assumed  by  [ 1 , Corollary  2(ii)l,  to  obtain  that 

(11)  (c  + v/\  )x  > 0 A 

n — n n 


can  be  "linearly  deduced"  (in  the  sense  of  [ 1 ])  from  (4).  (Note  that  an 
'x'  is  missing  in  ( 1 , Corollary  21). 

Changing  0 to  p in  (6),  (7)  of  our  Theorem  1,  to  avoid  confusion  with 


0 above  and  changing  in  (6),  (7)  to  cp  , to  avoid  confusion  with  X , our 


result  gives  (with  v = w) 


(11)' 


(c  + vA  )x 
n 


E tp.a^x  > E tp.b. 
iel  iel  ^ ^ 


> d + w /X  . 

- On 

Thus,  (11)^  can  be  linearly  deduced  from  (4).  Obviously,  by  taking 

0 = X d + W-,  from  [ 1 , Corollary  2(ii)]  we  obtain  [ 1 , Corollary  2(lli)]. 

n n u 

This  recovers  [ 1 , Corollary  2],  and  in  fact,  more:  for  any  sequence 

of  \ >0  with 

n 

(12)  lim  = +CD 

n 


1 


7 


we  have  seen  how  to  construct  0^  such  that  [ 1 , Corollary  2]  holds.  Thus, 

our  Theorem  1 changes  an  existential  statement  "there  exists  such  that..." 

into  a universal  statement  "for  every  X ...." 

n 

From  Corollary  2,  it  is  possible  to  obtain  the  main  result  of  [ l]. 

We  sketch  the  proof.  (Kortanek  [ 7 ] has  called  this  result  Blair's  "ascent 
ray"  theorem.) 

Theorem  2;  [ 1 ] 

If  the  system  (4)  is  inconsistent,  there  exists  a vector  w e r”  such  that, 
for  any  N,  the  inequality 

(13)  wx  N 
can  be  linearly  deduced  from  (4). 

Proof : The  inconsistency  of  (4)  shows  that,  for  (x,r)eR'^^,  xeR, 

a^x  - b.r  0 
1 — 

r > 0 

implies  -r  > 0. 

We  apply  Lemma  2 with  c = (0,-1),  and  find  that  there  exists 
(w,-Wq)  sr'^^,  weR*',  such  that  (dividing  (1)  by  9 on  both  sides): 

For  any  0 < 9 < 1,  there  is  a set  of  nonnegative  multipliers 
CXj^  I i 6 l},  only  finitely  of  which  are  nonzero,  and  cp  > 0 such  that 

(14)  (O.-1/0)  + (w,-Wq)  = cp(0,l)  + E X^(a^-b^). 

iel 

Taking  components  in  (14),  we  see  that 

WX  > 1/0  + Wq 


(15) 
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• A Constraint  Qualification  Weaker 
'limn  the  l^.lstence  of  ^ Vector 

The  convex  program  studied  in  this  paper  is 

inf  CqCx) 

(CP)  subject  to  I;  0 hell 

and  X e K 

where  H is  an  index  set  of  arbitrary  cardinality,  K is  a nonempty  convex  set, 
and  fj^  for  h etO)UH  maps  a convex  set  s K into  R,  and  Dj^  is  the  domain 

of  fj^.  (In  the  terminology  of  ^ ^ ^ ^h^^^  = + =»  for 

X ^ would  be  assumed  in  [ 9 ] . ) We  also  assume  that  relint(K)  c islint(Dj^) 
for  all  h€{o}UH,  where  relint(S)  denotes  the  relative  interior  of  the  set  S 

I 9 1. 

We  introduce  the  following  constraint  qualification  for  (CP): 

(CQ)  fhere  exists  x*^  in  the  relative  interior  relint(K)  of 
K such  that  fj^(x*^)  < 0 for  all  h eH. 

Note  that  (CQ)  is  quite  weak.  E.g.,  for  K = r”,  (CQ)  simply  asserts 
that  (CP)  is  consistent.  Thus,  (CQ)  is  satisfied  by  the  convex  program 

inf  (-y) 

(16) 

2 2 1/2 

subject  to  (x+y)  -x<0 

which  is  well-known  as  not  possessing  any  Kuhn-Tucker  vector.  I.e.,  letting 
v(P)  denote  the  value  of  the  primal  problem  (CP),  we  have  here  v(P)  = 0 
(since  the  constraints  have  solutions  (x,y)  = (x,0)  for  x > 0),  yet  there  is 
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no  scalar  > 0 with 

2 1 \ 12 

(17)  inf  -y  + - x]  > v(P)  = 0. 

X ,yeR 

In  fact,  for  any  > 0,  and  for  any  specific  y^  > 0,  by  choosing 

2 

Xq  = (^lyQ  ■ we  have 

(18)  -yo+ S«’'o+ ■ V ■ -yo+i- 

and  thus  the  infimum  in  (17)  is  - . For  X^^  = 0,  again  this  infimum  is  - <=  . 

The  purpose  of  tl^  constraint  qualification  (CQ),  is  simply  to  insure 
that  the  infimal  value  v(P)  of  (CP),  when  finite,  is  that  of 


inf  cl(fQ)(x) 

(CP)^  subject  to  cl(fj^)(x)  < 0 for  all  h cH 

and  X € cl (K) , 

where  cl(S)  is  the  closure  of  the  set  S s;  and  cl(f)  is  the  closure  of 
the  convex  function  f (see  [ 9 ]).  When  this  equality  of  value  holds,  since 
the  set  cl(K)  and  the  epigraphs  of  the  functions  cl(fj^)  for  he[0}UH,  can 
be  expressed  as  the  intersection  of  hyperplanes  in  r’^^  (see  [ 9 ]),  the 
techniques  of  the  preceding  section  can  be  applied  to  derive  results 
concerning  v(P).  From  these  motivational  remarks,  it  follows  that  (CQ)  could 
be  replaced,  in  our  results  in  the  next  section,  by: 

(CQ  y f for  h e { 0}  U H is  a closed  convex  function,  with 

real  values  in  the  relative  interior  of  the  convex 
set  2 K,  whose  only  non-real  value  is  + ® , with 
domain  dom(fj^)  lying  between  and  cl(Dj^);  and  there 
exists  x^’*^  e relint(Dj^)  with  fj^(x^’^)  < 0. 
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The  role  of  in  (CXJ)^  will  emerge  In  the  proof  (below)  of  gemma  4. 

To  piMvo  l lu“  claim  ma<U-  in  «)ur  motivutional  riMiiark , on  the  valne- 
ecpilvalence  t)l  (Cl*)  unci  (CP)^  nixli-r  (CQ),  we  establish  the  next  result. 

Tliis  reproduces  material  from  our  joint  paper  [ 6 | , both  results  and 
proofs,  with  the  kind  permission  of  our  co-author  R.J.  Duff in.  Since 
the  paper  [ 6 1 is  in  a preliminary  version,  there  appears  to  be  only 
this  course  of  action,  given  our  desire  to  make  this  paper  self-contained. 

Lemma  3 : [ 6 ] Let  the  value  v(P)  of  (CP)  be  finite,  and  suppose  that 

(OQ)  holds. 

Then  v(P)  is  also  the  value  (possibly  not  attained)  of  the  convex 
program 

inf 

(19)  subject  to  ^ hcH 

and  xerelint(K). 

Proof : For  each  n,  let  x^”^  be  chosen  to  insure 

(20)  fQ(x^"^)  < v(P)  + 1/n 

(21)  fj^(x^^^)  < 0 for  all  hcH 

(22)  x^’^^  e K. 

Let  x^erelint(K)  satisfy  fj^(x^)  < 0 for  all  heH.  Then,  for  any  X, 

0 <X  < I,  putting  u^*^^  = X,x^”^  + (1  -\)x^°^  we  have 

(23)  e relint(K) 

fj^(u^"^)  < 0 for  all  heH. 


and  by  convexity 
(24) 
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Now  is  continuous  on  the  line  segment  , with  in  fact 

(25)  ^ lim  + (1 I 0 < X < 1,  X ri], 

as  X c relint(K)  c relint(Dj^).  Hence,  for  X < 1 close  enough  to  unity, 

(26)  ^ 

< v(P)  + 2/n 

using  (20), 

From  (23),  (24)  and  (26),  we  see  that  the  value  of  the  program  (19) 
does  not  exceed  v(P).  On  the  other  hand,  the  program  (19)  is  more  constrained 
than  (CP),  so  its  value  cannot  be  less  than  v(P).  Thus,  its  value  is  exactly 
v(P). 

Q.E.D. 

Our  proof  of  Lemma  3 also  can  be  modified  to  reveal  that,  if 
v(P)  = - , then  - ® is  also  the  value  of  (19). 

Corollary  1:  If  v(?)  is  finite  and  (a))  holds,  then  the  value  of  (CP)  is 
also  that  of  (CP)^  . 

Proof:  We  recall  [9]  that  relint(cl(K))  = relint  K,  and  that,  for  all 
hc[0}UH,  fj^(x)  = (cl  fjj)(x)  for  all  xerellnt  K,  since  relint  K = relint 
is  assumed . 

Thus,  when  we  start  with  the  program  (CP)^  viewed  as  (CP),  and  construct 
the  analogous  program  (19),  we  in  fact  end  up  exactly  with  (19). 

By  Lenina  3,  both  (CP)  and  (CP)^  have  the  value  of  (19),  hence  both  have 
value  v(P). 


Q.E.D. 
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Section  3:  Hie  Main  Result 


Our  nuiln  result  ('l'liei>rem  3)  Is  obtained  l)y  upi^lyiiiR  'llieorem  I to  a 
"semi -inf ini te"  system  of  linear  inequalities  equivalent  to  (CP)^  , and 
then  Interpreting  this  outcome  by  methods  of  algebraic  manipulation 
developed  in  [ 6 ] . We  reproduce  the  latter  here,  for  the  sake  of  a self- 
contained  presentation. 

By  Corollary  1,  we  may  assume,  within  our  proofs,  that  K is 
closed,  as  are  the  functions  fj^  for  hc{0}UH.  This  is  assumed  throughout 
the  remainder,  ilierefore,  we  have  representations  via  hyperplanes; 

(27)  K = {x  cr"  I a^x  2 aj,  J c I(-l)} 

(28) |^  epi(fj^)  = {(z,x)  e 8”*"^  X c r"  I b^z  + a^x  > a^,  jel(h)] 

for  index  sets  1(h),  h€[-l}  U {0}  U H,  where  possibly  I(-l)  = 0,  but  1^  ^ 1) 
for  h -1,  and  as  usual  epi(fj^)  denotes  the  epigraph  of  fj^: 

(29)  epi(fj^)  = {(z,x)  e X e I z > fj^(x)} 

which  is  a closed,  convex  set.  Obviously,  in  (28)^^,  bg  > 0 for  all  j e 1(h) 
and  h e {O]  U H. 

Without  loss  of  generality,  the  representations  (27),  (28)^  consist 
precisely  of  all  supporting  hyperplanes  for  the  closed,  convex  sets  K and 

epi(fj^),  h e {0}UH. 

f 

I. 

Lemma  4:  [ 6 ] Fix  h cH,  and  suppose  that  (CQ)  holds. 

Then  for  any  xcDj^,  < 0 is  equivalent  to  the  semi-infinite  system 


> a^,  j c 1(h). 


(30) 
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I 

f 

i 

i 


I 


I 


Proof:  If  5 since 

we  have 


(fj^(x),x)  e epi(fj^)  and  > 0 for  all  j el(h), 


a^x  - a^  + (-b^)fj/x)  > aj 
whenever  x * Dj^  and  j el(h). 

For  the  converse,  suppose  (30)  holds  for  x (i.e.,  a^x  > a^  for  all 
j cl(h))  yet  > 0.  By  (CQ),  there  exists  x®erelint(K)  = relint(Dj^) 

with  fj^(x^)  < 0.  Without  loss  of  generality,  fj^  is  closed.  Therefore,  it 
is  continuous  on  line  segments  and,  denoting  x(A.)  = A.x  + (1  -A,)x^,  0 < A.  < 1, 
we  have  = lim  {x(A)  | X ^l}.  Thus,  for  some  X*,  with  0 < A*  < 1, 

fh(x(X*))  > 0. 

For  all  A in  the  range  0 < A <1,  x(A)  €relint(K)  c relint(Dj^),  and 
hence  there  exists  a subgradient  -u  e r”  to  fj^  at  x(A*),  i.e., 

(32u)  »*(x(a*)  - x)  for  all  xcU|^. 


From  £j^(x(A*))  • 0 and  £j|(x^)  = rj^(x(0))  < 0,  we  obtain  from  (32a)  (with  x = x^), 

0 > fj^(x(A*))  + A*u(x-x^)  and  hence  u(x  - x*^)  < 0.  This  in  turn  implies  that 
u(x  - x(A*))  = u(x  - A*x  - (1-A*)x°)  = u(l  - A*)(x  - X®)  < 0.  Thus,  since 
fh(x(A*))  > 0,  the  inequality  ux  > fjj(x(X*))  + ux(A*)  cannot  hold. 

Now,  since  (32a)  holds,  without  loss  of  generality, 

(32b)  z + ux  > fj^(x(X*))  + ux(X*) 


is  among  the  defining  inequalities  in  the  system  (28)j^.  I.e.,  for  some 

J €l(h),  b^  - I,  a^  = u,  and  a^  = fj^(x(X*))  + ux(X*).  However,  since 
ux  < fj^(x(X*))  + ux(X*),  as  we  saw  in  the  last  paragraph,  for  this  choice 
of  j el(h)  we  have  a-^x  < a^,  and  this  contradicts  (30)  and  completes  the 
proof . 

Q.E.D. 


t 


,t 


1 

I 


.1 
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Corollary  2:  (A  la  [6]). 

Assume  that  (CQ)  liolcls. 
if  finite,  is  also  the  value 


'Ihen  the  value  v(P)  of  the  convex  program, 
of  this  semi -inf ini te  program; 


(SI) 


subject  to 


inf  z 

b^z  + a^x  _•  a^,  j € I (0) 

a^x  > a^  for  all  j el(h) 
and  h e (-l]UH. 


Proof : Imnediate  from  Lemma  4. 


Q.E.D. 


It  is  now  the  point  to  complete  the  program  outlined  at  the  beginning 
of  this  paper,  and  invoke  Theorem  1.  This  program  is  identical  in  conception 
to  that  of  our  joint  paper  [ 6 ] , which  differed  only  in  that  a different 
and  stronger  result  on  the  semi-infinite  system  (SI)  was  invoked,  which  was 
possible  in  I 6 1 because  a constraint  qualification  stronger  than  (CQ)  was 
assumed,  which  led  to  an  (SI)  with  some  special  properties. 


Theorem  3:  Suppose  that  the  constraint  qualification  (CQ)  holds  for  the 
convex  program  (CP)  of  finite  value  v(P). 

Then  there  exists  Wq,  Wj^  e R and  w c r”  with  the  following  property: 

For  any  scalar  0 in  the  range  0 < 0 < 1,  there  exist  y Y^«R>  and 

nonnegative  scalars  1 hen),  only  finitely  many  of  which  are  nonzero,  and 
« r”,  P q « R for  h e (0}  U H,  satisfying: 

I 
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Condition  1:  Yx  + Yq  1:  ^ for  x e K 

Condition  2:  + p ^ for  all  x e Dj^  and  h e {0}  U H 

Condition  3: 

Yx  + Y°  + (1+0Wq)(P°x  + Pq)  + 0(wx  + w^)  + E \^(p\  + p^)  > v(P) 

heH 

for  all  X c r'^'. 

In  fact,  Wq,  Wj^,  w can  be  chosen  arbitrarily  to  satisfy 

(33)  (Wq.w.Wj)  = (Vq.v.v^)  - (1,0, -v(P)) 

where  (Vq.v.Vj^)  (with  ,V2  e R and  v e R*')  is  any  point  In  the  relative 
interior  of 

(34)  C = cone  ({(b^.a-*  ,-a^)  | j e 1(0)]  U U [ (O.a-^ , -a^)  | j € I (h)] 

° ° he{-l}UH  ° 

U {(0,0,1)}). 

Proof : By  Corollary  2 and  by  Hieorem  1,  equations  (6)  and  (7)  (which  we 
saw  is  equivalent  to  (10)),  we  have,  for  0 < 9 < 1, 


(35)  (1,0, -v(P))  + 0(Wq,w,w^)  = cp(0,0,l) 
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to  (33),  such  that  (Vq.v.Vj^)  is  in  the  relative  interior  of  C of  (34), 
Wo  now  analyze  (35)  by  the  methods  of  [ 6 ]. 

From  the  first  components  in  (35),  we  obtain 


(36) 


1 + 9w„ 


j€l(0) 

bj>0 


We  will,  in  general,  define  for  hc{0]UH 


(37) 


j 

ro 


E 9.b 

jel(h) 

bl>0 


(with  the  understanding  that  = 0 if  b^  = 0 for  all  j el(h)  with  cpj  > 0) 

for  h c{0]UH.  Thus  we  have  \q  = 1 + 9wq  from  (36).  Clearly,  only  finitely 

many  \ can  be  nonzero,  and  all  > 0,  by  the  conditions  on  the  scalars 
h h — 

j 


cp  > 0. 


Next  we  define  these  vectors  and  scalars ; 


(38a) 


Y = 


- E 9,3"^  - ^ 2 91®^ 

jel(-l)  ^ he{0]UH  jel(h)  ^ 

bJ-O 


(38b) 


Yn  “ 


° j«I(-l)  ° hcC0}UH  Jel(h)  ° 


bJ.O 


where  an  empty  summation  is  zero.  From  (27),  if  J cl(-l),  a^x  > a^  for 
X cK;  and  from  (28)^^,  if  b^  = 0,  we  have  again  a^x  > a^  for  J cl(h)  and 
h e {0}  UH,  as  X cDj^  2 K.  Hence  y*  + Yq  5 ® x * K,  i.e. , Condition  1 
holds. 
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For  h e[0}UH,  if  bj^  ^ 0,  since  bQfj^(x)  + a^x  > for  xeD^^,  we  have 


(3‘)) 


Combining  (37)  and  (39)  for  Aj^  (),  we  have 


(40) 


j.i(h)  J 


1>J>0 


> Z ;p  bh-  ^ X + a?/bh,  for  xeD.  , 
4^T/Vi\  J ^ K-l  u n 


jel(h) 

bj>0 


and  so  defining 


(41)  + ^ E cp  bj(-  ^ X + aj/bj) 

jel(h)  ^ bj 

bJ>0 

we  obtain  Condition  2.  To  be  precise,  we  actually  have  cl(fj^)(x)  > 3^x  + 8q 
for  all  xcDj^,  but  since  cl(fj^)(x)  < fj^(x)  for  all  xsDj^,  Condition  2 follows. 

If  = 0,  one  can  arbitrarily  pick  p^x  + 3^  to  satisfy  Condition  2,  at 
least  one  such  affine  form  existing  since  f^^  is  somewhere  finite,  and  hence 
has  at  least  one  subgradient  at  one  point. 

For  the  concluding  part  of  our  analysis,  we  write  (35)  again  with 
the  first  component  dropped,  in  this  form: 


(35)' 


- 9(0.1) 


+ (-Y.-Yn) 


+ E E cp  bJ(aVbJ,-aJ/bJ) 

hc{0}UH  jel(h)  J ° 0 0 0 

bJ>0 


(0,-v(I'))  b 0(w.wp 
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= 9(0.1)  + (-y,-Yq) 


+ 


he{0}Ull 


It  remains  only  to  dot  product  both  sides  of  (35)^  with  (-x,-l), 
where  x c is  arbitrary,  to  obtain 


(42) 


v(P)  + 0(-wx  -wp  = -cp  + (YX  + Yq) 


he{0}UH 


Xh®"*  + pj) 


for  all  X eR  . Since  9 > 0,  (42)  immediately  yields  Condition  3,  using 
^0=1+  Owq. 

Q.E.D 

Coro [ lary  3 : Suppose  that  the  constraint  qualification  (CQ)  holds  for  the 
convex  program  (CP)  of  finite  value  v(P). 

Then  there  exists  Wq,  Wj^  e R and  w e R*^  with  the  following  property: 

For  any  scalar  in  the  range  0 < 0 < 1,  there  exist  scalars  | h cH}, 
only  finitely  many  of  which  are  nonzero,  with 


(43)  (1  + 9wQ)fQ(x)  + 0(wx  + w^)  + E X^f^(x)  > v(P) 

heH 

for  all  X c K. 

Furthermore,  Wq,  Wj^  and  w can  be  arbitrarily  chosen,  subject  to  the 
condition  (33),  where  (Vq,v,Vj^)  is  a point  in  the  relative  interior  of 
the  convex  set  C of  (34). 

Proof:  Immediate  from  Theorem  3. 
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Section  A:  Relationship  to  the  Usual  Lagrangean 

We  conclude  by  attending  to  two  matters,  which  are  not  of  great 
technical  difficulty,  but  which  do  serve  to  clarify  the  nature  of  our 
main  result  (Iheorem  3),  more  specifically  its  corollary  (Corollary  3), 
and  how,  in  a limiting  sense.  Corollary  3 solves  (CP).  This  clarification 
Involves  some  specification  on  the  vectors  (Wq,w,Wj^)  of  Corollary  3 in 
certain  cases . 

1 
i 

In  the  usual  Lagrangean  and  its  associated  Kuhn-Tucker  theory, 
typically  one  seeks  sufficient  conditions  for  the  equality, 

(44)  max  inf  {ff,(x)  + E X,  f.  (x)}  = v(P), 

Xj^>0  xeK  hcH 

heH 

where,  in  some  instances,  the  "max"  is  relaxed  to  a "sup"  (supremum). 

The  usual  tlieory  as  developed  in  | 9 I re<julres  the  cardinality  of  II  to  be 

finite.  As  wo  saw  in  Section  2,  our  constraint  qualification  (OQ)  does 

not  even  insure  (44)  with  "max"  replaced  by  "sup." 

Our  additional  conditions  D,  2 K and  relint (D.)  2 relint (K)  are  also 

h h 

typical  of  the  standard  treatment  of  "ordinary  convex  programs,"  as 
presented  in  [ 9 ],  and  in  fact  [ 9 ] requires  Dq  = K,  which  we  do  not. 

(Restricting  a function  f^  from  a domain  Dq  ^ K to  K will  typically 
significantly  affect  its  subgradients.  E.g.,  a one -dimensional  differentiable 
convex  function  defined  on  R has  a unique  subgradient  at  x > 0,  but  if  f is 

restricted  to  K = {x  | x > 0}  it  has  many  subgradients  at  zero.  It  is 

essentially  due  to  this  restriction  Dq  = K,  that  no  term  of  the  form 

yx  + Yq»  nonpositive  on  K,  appears  in  certain  results  of  [ 9 ),  such  as 

[ 9 , Theorem  28.3  and  particularly  Theorem  28.3(c)J. 
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What  our  Corollary  3 concerns,  is  relations  more  complex  than  (44), 
due  to  the  presence  of  one  more  operation  on  the  left -hand -side:  the 
taking  of  a limit. 

In  what  follows  A denotes  all  multipliers  | heH)  which  are 
nonnegative  and  finitely  nonzero.  We  next  establish  a crucial  inequality. 

Lemma  5:  Suppose  that  (Wq(0) ,w(0) ,Wj^(0))  e w(0)cr’^,  is  defined  for 

0 < 0 < 1,  and  the  set  of  all  vectors  of  this  form  is  bounded.  Then  if 
(CP)  is  consistent  and  has  finite  value  v(P), 

(45)  lim  sup  sup  inf  { (1 + 0w„(0))f „(x)  + 0(w(0)x  + w. (0)) 

0 \ Q-’’  A xeK 

+ E < v(P) 

heH 

Proof : First  observe  that  for  any  0,  and  element  of  A, 


(46) 


inf  [(1  + 0WQ(0))fQ(x)  + 0(w(0)x  + w^(0))  + E 
xeK  heH 

< inf  C(1  + 0WQ(0))fQ(x)  + 0(w(0)x  + w^(0))  I fj^(x)  < 0,  heH} 
xeK 


since  all  > 0 if  (Xj^  | hcH)  e A.  Therefore,  the  left -hand -side  in  (45) 
docs  not  exceed 

(47)  lim  siw  inf  { (1  + 0w  (0))f q(x)  + 0(w(0)x+ Wj(0))  | fj^(x)<0,  heH} 

0 (r  xeK 

Next,  let  x^*'^  be  chosen  so  that  fj^(x^”^)  < 0 for  heH  and 
fg(x^"^)  < v(P)  + 1/n,  which  is  possible  since  v(P)  is  the  value  of  (CP). 
We  see  that  (47)  does  not  exceed 
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(48) 


C(l+ OwQ(9))fo(x^’'^)  + 9(w(9)x^"-'  + w^(9))} 


(n) 


- V(l’)  I I/ll, 


using  the  boundedness  condition  on  (Wq(9) ,w(9) ,Wj^ (9) ) . Now  if  v(P)  + 1/n 
is  an  upper  bound  on  the  left-hand-side  of  (45)  for  any  n,  so  is  v(P). 
This  establishes  (45). 

Q.E.D. 

From  (43)  of  Corollary  3,  for  any  9 in  the  range  0 < 9 < 1 we  have 


(49) 


sup"^  inf  t(l+9w  )f_(x)  + 9(wx  + w.)  + S X f (x)}  > v(P) , 

A xeK  heH 


where  a statement  sup  | jej}  > a for  us  abbreviates  the  condition 

that  > a for  some  j e J,  and  hence 


(49)' 


lim  inf  sup'*' inf  { (1  + 9w  )f  (x)  + 9(wx+ w, ) + Z X,f.(x)}  > v(P). 

9 \ 0^  A xeK  ^ hcH  " " 


Putting  together  (45)  of  Lenina  5 and  (49)'  above,  we  obtain  (under 
the  hypotheses  of  Corollary  3) 

(50)  lim^sup'^inf  { (1  + 0wQ)f  ^(x)  + 9(wx  + Wj)+  E Xj^fj^(x)]  = v(P). 

9 \ 0 A xeK  hcH 

Comparing  the  standard  Lagrangean  result  (44)  with  ours  (50),  we  see  their 
similarity  in  nature.  The  limit  appearing  in  (50)  suggests  the  term 
"limiting  Lagrangean"  for  our  results. 

We  now  state  a way  in  which  our  limiting  Lagrangean  can  be  used  to 
solve  (CP),  in  a limiting  sense. 

Lemma  6:  Suppose  that  (Wjj(O)  ,w(0)  ,Wj(0))  e w(0)cr",  is  defined  for 

0 < 0 < 1,  and  the  set  of  all  vectors  of  this  form  is  bounded.  Let  the 


value  v(P)  of  (CP)  be  finite. 


Let  0^  in  the  range  0 < 0^  < 1 satisfy 


lim  0 = 0 . 

n 

n 


Suppose  also  that  the  functions  fj^,  he[0}UH  are  continuous,  and  that  for 
each  n the  vector  satisfies,  for  certain  | heH)  e A, 

(52)  v(P)  inf  1(1+ 0^WQ(0^))fQ(x)+0^(w(0^)x+w^)+  E x"f^(x)} 

xeK  heH 

h«H 

with  lim  sup  a <0 
n n - 


f,  (x^*^^ ) < cr^ , with  lim  sup  < 0 for  all  h c H ; 
h - n n - 


E x[)f,  (x^”^)  = p where  lim  sup  p = 0. 
heH  " n 


Then  if  tlie  sequence  of  x'  has  a limit  point  x*,  x*  will  be  an 
oiitlimmi  to  (Cl’). 

Proof:  By  continuity,  as  we  may  assume  lim  x^*'^  = x*,  we  have 

n 

f(x*)  = lim  f,  (x^*^^)  < lim  sup  < 0,  so  x*  is  feasible  in  (CP), 
n " n " 

By  virtually  repeating  the  proof  of  Lemma  5,  we  obtain,  as  lim  sup  a <0, 

n 

(55)  lim  sup  sup  ((l+9^Wg(0^))fQ(x^”^)+0^(w(0^)x^"^  + w (0^)) 


+ E X.f.(x^"^)}  < v(P). 


Using  (51),  (54),  and  (55)  we  obtain  (via  the  boundedness  assumption) 
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v(P)  :r  lim  sup  {(l+0^WQ(9^))f(j(x^"^+9^(w(9^)x^’"Uw^(9^))} 


= 1 (x*) . 


Thus,  C(x*)  is  optimal  in  (CP)' 


Q.E.D. 


As  regards  the  determination  of  e R and  w c R of  Theorem  3 and 
Corollary  3,  we  first  discuss  a few  points  concerning  the  determination 
of  interior  points  (Vq,v,Vj^)  of  C in  (34),  We  do  not  try  here  to  give  an 
efficient  algorithm;  we  merely  wish  to  indicate  that  these  quantities  are 
often, in  principle,  computable.  Since  v(P),  the  determination 

of  Wj^  involves  a knowledge  of  the  value  v(P)  of  the  program  (CP),  and 
hence  is  a more  complex  matter.  We  shall  make  remarks  about  w^^  of  (33) 
in  our  concluding  statements. 

For  vectors  v^  e R^  and  a nonempty  index  set  1 0,  the  determination 

of  an  interior  point  of 

(57)  Cj.  = cone  ({v^  | iel}) 

is  never,  in  principle,  problematic,  once  one  has  some  spanning  set,  say 
{v^,...,v^}  of  (v^  I iel],  in  the  sense  of  a vector  space  span.  An  interior 
point  of  of  (57)  is  always  given  by 

(58)  V = v^  + . ..  + v*^. 


Indeed,  for  any  vector  w = T,  p.v  in  the  vector  space  spanned  by 

i=l  ^ 

[v^,.,.,v^},  i.e.,  in  the  manifold  spanned  by  of  (57),  there  is  a 
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sufficiently  small  e > 0 so  that 

(59)  1 + ep^  0 for  i = 1 t 

;u«l  lic‘iic«'  V -I-  cw  C C I . Sinci'  w in  llio  iiwiniroUl  s]mnned  by  was 
arbitrary,  v of  (58)  is  an  interior  point  of  by  standard  criteria 
(see  c.g.,  I 9 J). 

To  obtain  {v^ , . . . .v*"] , it  often  suffices  to  know  t,  the  dimension 
of  the  manifold  spanned  by  Cj.  Here  we  have  in  mind  primarily  the  case 
that  a countable  dense  subset  {v^  | iel},  which  can  be  effectively  listed, 
can  be  effectively  extracted  from  {v^  | iel}.  Since  clcone  (Cv^  | iel^}) 
s cone  ({v^  | i«l}),  there  are  also  t linearly  independent  vectors  in 
{v^  I itl^].  Thus  one  can  simply  continue  a listing  until  t independent 
ones  are  found.  We  avoid  details  on  the  points  raised  in  this  paragraph, 
since  a full  discussion  of  these  matters  requires  a knowledge  of  recursion 
theory,  which  we  do  not  assume  here. 

Tlie  simplest  case  is  t = q,  i.e.,  the  cone  of  (57)  is  fully- 
diincnslonal , and  our  next  result  shows  that  this  is  indeed  a very  common 
case  for  the  cone  C of  (34).  Before  we  begin  the  proof  of  our  next 
result,  one  may  remark  that  full -dimens tonality  occurs  if 

(60)  v^x  = 0 for  all  iel  implies  x = 0. 

Indeed,  if  (60)  held  but  the  linear  span  of  {v^  | iel}  was  a subspace 
L ^ R^,  its  perpendicular  subspace  l"  has  a nonzero  vector  x.  Then 
v^x  = 0 for  all  iel  but  x 0,  a contradiction. 

I 

Lemma  7:  Suppose  that  (CP)  is  feasible  are  has  finite  value  v(P). 

Barring  the  case,  that  there  exists  a nonzero  vector  x*  such  that  for 


(62a) 


f|^(x  + Ox*)  < 0 all  h e H; 


(62b) 


f (x  + Ox*)  = £(x); 


(62c) 


X + Ox*  e K; 


then  the  cone  C of  (34)  is  of  full  dimension  (n+2). 


Proof : We  have  to  show  that  there  is  no  nonzero  solution  to  all  the 


equalities 


b^z  + a^x  - a^w  = 0,  j el(0) 


a^x-a^w  = 0,  jel(h)  and  he{-l}UH 


w = 0 


or,  equlvalontJy , to  the  e<|ualltles 


b^z  + a^x  = 0,  j e 1(0) 


= 0 , j c 1 (h ) and  he{-l)UH. 


Since  at  least  one  b^  > 0 for  j el(0),  as  f^  has  a subgradient  at  at 


least  one  point,  we  cannot  have  x » 0 in  a nonzero  solution  to  (63)^  . 


Suppose  that  (z*,x*)  solves  (63)'  , so  that  x*  0.  By  homogeneity. 


we  may  assume  z*  < 0.  Let  x be  any  solution  to  (CP).  Then  for  any 


0 > 0,  fixing  h c {-l}UH  and  letting  J c 1(h)  be  arbitrary,  we  have 


a-^(x  + 0x*)  * a^x  + Qa^x* 


= a-^x  > a^ 


since  a'^x*  ■ 0 from  (63)^  and  a'^x  > a^  by  Lemma  4.  Thus,  by  Lemma  4, 
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wi'  liuvo  unci  (h'J.c)  I'or  H 0.  Alm»,  .since  (x)  I-  a'^x  nj^  lor 

any  ) e I (»)  (a.s  (l||(x),x)  we  liave 

(65)  b^(fQ(x)  + 9z*)  + a-^(x  + 9x*) 

= (b^fQ(x)  + a^x)  + OCb'^z*  + a-^x*) 

= b^fgCx)  + a^x  > 

since  b^z*  + a'^x*  = 0.  This  gives  (62b)  for  9 > 0,  with  = replaced 

by  <,  since  (fQ(x)  + 0z*,  x + 0x*)  e epi(fQ)  and  z*  < 0,  fQ(x)  + 0z*  < f (x) 

for  all  0 > 0. 

Now  if  z*  ' 0,  from  the  above,  fQ(x  + Ox*)  can  be  indefinitely 
decreased  by  sending’  0 /“+■>’,  and  all  the  while  x + 0x*  is  feasible  In 
(CP)*  lliis  contradicts  that  CP  has  finite  value.  Hence  z*  = 0,  and 
we  can  repeat  the  analysis  with  (“Z*,-x*)  replacing  (z*,x*),  and  in  this 
manner  obtain  (62a)  and  (62c)  for  all  9eR.  We  also  obtain  fQ(x  + 0x*)  < fQ(x) 
for  all  0eR;  since  f^  is  convex,  we  clearly  have  (62b). 

Q.E.D. 

Thus,  if  it  is  known  that  the  feasible  region  contains  no  full  line, 
or  that  |f(x)|  7 + » as  l|x||  T , or  that  f^  is  not  constant  on  any 
line,  or  that  f^  is  not  constant  on  any  line  in  the  feasible  region  of  (CP) 

— all  of  these  being  commonly-occurring  hypotheses  — Lemma  7 shows  that 
the  dimension  of  the  cone  C of  (42)  is  full,  i.e.,  is  (n+2). 

From  (33),  once  an  interior  point  (Vq,v,Vj^)  is  found,  we  can  compute 
(Wq,w,Wj)  for  use  in  the  limiting  Lagrangean  by  Wq  ■ Vq-  1,  w « v,  and 
> v^  + v(F).  Only  the  last  equation  is  problematic,  since  it  appears  to 
involve  an  exact  knowledge  of  the  value  v(P)  of  the  convex  program  (CP). 


i 


28 


However,  an  inspection  of  (43)  sliuws  that,  if  Wj  replaces  w, 

in  the  limiting  lAgrangean,  we  will  still  obtain  a limiting  l^agrangean, 
as  0 0.  llierefure  it  is  necessary  only  to  know  a bound  M on  tlie  value 

of  (CP),  and  since  v(P)  < M,  we  may  set  + M. 

Often  such  bounds  M are  obtained  from  feasible  solutions  to  (CP). 

In  Jiny  event,  since  M can  be  set  most  liberally,  even  the  most  cursory 
information  about  (CP)  will  allow  one  to  compute  w^  from  Vj^. 

A few  final  remarks  are  in  order.  The  limiting  Lagrangean  can,  of 
course,  reduce  to  the  ordinary  one,  if  (l,0,-v(P))  is  in  the  relative 
interior  of  the  cone  C of  (34),  for  then  we  get  (w^^.w.Wj^)  = 0 in  (33). 
However,  it  is  possible  that 

(66)  (1,0, -v(P))  e C 

but  not  (1,0, -v(P))  e rellnt  C (recall  that  always  (1,0, -v(P))  e cl(C)), 
in  which  case  the  limiting  Lagrangean  does  not  reduce  to  the  ordinary 
one. 

Nevertheless,  in  this  latter  case,  one  easily  sees  that  the  term 
0(Wq,w,Wj^)  can  be  omitted  in  (35),  and  then  if  the  analysis  in  the 
remainder  of  the  proof  of  Theorem  3 is  repeated,  we  obtain  the  usual 
Lagrangean- type  relationship  (44).  We  say  "Lagrangean- type"  rather  than 
"Lagrangean,"  since  H can  be  infinite,  yet  only  finitely  many  of  the  Xj^, 
h eH  will  actually  be  nonzero  (l.e.,  all  but  finitely -many  of  the 
constraints  of  (CP)  can  be  omitted  without  changing  its  value). 

TC  get  Lagrangean  results  of  the  usual  type  (44),  one  needs  to  know 
when  (66)  holds.  In  this  regard,  we  have  Jointly  verified  [6  ] that 
many  of  the  "constraint  qualifications"  (such  as  those  of  [ 9 , Theorem  28.2] 
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and  others)  are  sufficient  conditions  for  the  cone  C of  (34)  to  be  closed, 
provided  that  the  "correct"  defining  Inequalities  are  chosen  In  equations 
(27),  (28)j^.  (An  "arbitrary"  representation  of  K or  epl(fj^),  which  simply 
defines  the  correct  set  of  points,  usually  will  not  do:  the  representation 
plays  a role  as  Important  as  the  actual  set.  For  a similar  circumstance 
where  this  Issue  arose,  see  [ 3 ] and  [ 5 ] .) 

The  closure  of  C trivially  Implies  (66),  by  Lemma  1,  as  (1,0, -v(P))  e cl(C). 
And  since  a "constraint  qualification"  Is  a condition  on  the  constraints 
alone,  allowing  the  objective  function  f^  to  vary  over  all  convex  functions. 

It  is  not  surprising  that  the  closure  of  C Is  Implied  by  these  hypotheses. 

More  can  be  said  on  these  matters ; see  [ 6 ] for  further  details  and  results . 


April  26,  1978 
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’ In  fact,  our  main  result  (Theorem  3 of  Section  3)  reveals  much  finer  detail 
’ concerning  our  "Limiting  Lagrangean."  There  are  affine  minorants  (for  any  ! 
value  0 < 9 < 1 of  the  limiting  parameter  9)  of  the  given  convex  functions,  ! 

plus  an  affine  form  nonpositive  on  K,  for  which  a general  linear  Inequality  , 

^ holds  on  R".  After  substantial  weakening,  this  Inequality  leads  to  the 
conclusions  of  the  previous  paragraph. 
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This  work  is  motivated  by,  and  is  a direct  outgrowth  of,  research  joint  with  ' 
R.J.  Duffin,  which  is  cited  as  our  reference  [6]. 
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